Exclusive Supermask Subnetwork Training for Continual Learning

Continual Learning (CL) methods focus on accumulating knowledge over time while avoiding catastrophic forgetting. Recently, Wortsman et al. (2020) proposed a CL method, SupSup, which uses a randomly initialized, fixed base network (model) and finds a supermask for each new task that selectively keeps or removes each weight to produce a subnetwork. They prevent forgetting as the network weights are not being updated. Although there is no forgetting, the performance of SupSup is sub-optimal because fixed weights restrict its representational power. Furthermore, there is no accumulation or transfer of knowledge inside the model when new tasks are learned. Hence, we propose ExSSNeT (Exclusive Supermask SubNEtwork Training), that performs exclusive and non-overlapping subnetwork weight training. This avoids conflicting updates to the shared weights by subsequent tasks to improve performance while still preventing forgetting. Furthermore, we propose a novel KNN-based Knowledge Transfer (KKT) module that utilizes previously acquired knowledge to learn new tasks better and faster. We demonstrate that ExSSNeT outperforms strong previous methods on both NLP and Vision domains while preventing forgetting. Moreover, ExSSNeT is particularly advantageous for sparse masks that activate 2-10% of the model parameters, resulting in an average improvement of 8.3% over SupSup. Furthermore, ExSSNeT scales to a large number of tasks (100). Our code is available at https://github.com/prateeky2806/exessnet.Comment: ACL Findings 2023 (17 pages, 7 figures

A Bayesian perspective on sampling of alternatives

In this paper, we apply a Bayesian perspective to the sampling of alternatives for multinomial logit (MNL) and mixed multinomial logit (MMNL) models. A sampling of alternatives reduces the computational challenge of evaluating the denominator of the logit choice probability for large choice sets by only using a smaller subset of sampled alternatives including the chosen alternative. To correct for the resulting overestimation of the choice probability, a correction factor has to be applied. McFadden (1978) proposes a correction factor to the utility of each alternative which is based on the probability of sampling the smaller subset of alternatives and that alternative being chosen. McFadden's correction factor ensures consistency of parameter estimates under a wide range of sampling protocols. A special sampling protocol discussed by McFadden is uniform conditioning, which assigns the same sampling probability and therefore the same correction factor to each alternative in the sampled choice set. Since a constant is added to each alternative the correction factor cancels out, but consistent estimates are still obtained. Bayesian estimation is focused on describing the full posterior distributions of the parameters of interest instead of the consistency of their point estimates. We theoretically show that uniform conditioning is sufficient to minimise the loss of information from a sampling of alternatives on the parameters of interest over the full posterior distribution in Bayesian MNL models. Minimum loss of information is, however, not guaranteed for other sampling protocols. This result extends to Bayesian MMNL models estimated using the principle of data augmentation. The application of uniform conditioning, a more restrictive sampling protocol, is thus sufficient in a Bayesian estimation context to achieve finite sample properties of MNL and MMNL parameter estimates

Indian Vehicle Ownership: Insights from Literature Review, Expert Interviews, and State-Level Model

This study reviews existing vehicle ownership models for India and describes the results of nine experts’ interviews to gather insights about Indians’ travel patterns and vehicle choices. According to the experts, vehicle price, fuel economy, and brand (in declining importance) are the most decisive factors in Indians’ car purchase choices. This study also estimated household vehicle ownership levels across India’s 35 states using Census 2011 data. The results suggest that states with a higher proportion of computer-owning households and higher share of households living in rural areas with larger household size, ceteris paribus, are likely to have higher car ownership

A Deep Generative Model for Feasible and Diverse Population Synthesis

An ideal synthetic population, a key input to activity-based models, mimics the distribution of the individual- and household-level attributes in the actual population. Since the entire population's attributes are generally unavailable, household travel survey (HTS) samples are used for population synthesis. Synthesizing population by directly sampling from HTS ignores the attribute combinations that are unobserved in the HTS samples but exist in the population, called 'sampling zeros'. A deep generative model (DGM) can potentially synthesize the sampling zeros but at the expense of generating 'structural zeros' (i.e., the infeasible attribute combinations that do not exist in the population). This study proposes a novel method to minimize structural zeros while preserving sampling zeros. Two regularizations are devised to customize the training of the DGM and applied to a generative adversarial network (GAN) and a variational autoencoder (VAE). The adopted metrics for feasibility and diversity of the synthetic population indicate the capability of generating sampling and structural zeros -- lower structural zeros and lower sampling zeros indicate the higher feasibility and the lower diversity, respectively. Results show that the proposed regularizations achieve considerable performance improvement in feasibility and diversity of the synthesized population over traditional models. The proposed VAE additionally generated 23.5% of the population ignored by the sample with 79.2% precision (i.e., 20.8% structural zeros rates), while the proposed GAN generated 18.3% of the ignored population with 89.0% precision. The proposed improvement in DGM generates a more feasible and diverse synthetic population, which is critical for the accuracy of an activity-based model

Spatial modeling of electric vehicle ownership across Texas and a simulation-based framework to predict Americans' adoption of autonomous vehicle technologies

textThis thesis is divided into four parts. The first part investigates the impact of built-environment and demographic attributes on adoption rates of hybrid electric vehicles and more fuel efficient vehicles. To allow for spatial autocorrelation (across census tracts) in unobserved components of tract-level vehicle counts, as well as cross-response correlation (both spatial and aspatial), vehicle counts by vehicle type and fuel economy levels were estimated using a bivariate and trivariate Poisson-lognormal conditional autoregressive model. Fuel-efficient-vehicle ownership rates were found to rise with household income, resident’s education levels, and the share of male residents, and fall in the presence of larger household sizes and higher jobs densities. In the second part, a fleet evolution framework is proposed to simulate Americans' long-term (year 2015 to 2045) adoption of connected and autonomous vehicle (CAV) technologies under eight-different scenarios based on: 5% and 10% annual drop in technology-prices; 0%, 5%, and 10% annual increment in Americans' willingness to pay (WTP); and NHTSA's regulations. A survey was designed and disseminated to obtain 2,167 Americans' preferences; and those data were used in simulation framework. The survey results indicate that Americans' average WTP (of the respondents with a non-zero WTP) to add connectivity and Level 3 and Level 4 automation are $110,$5,551, and $14,589, respectively. The simulation results suggest that 24.8% (at 5% drop in technology-prices and constant WTP) to 87.2% (at 10% drop in technology-prices and 10% WTP rise) of the Americans' privately owned vehicle-fleet will be fully-automated by 2045. The parts three and four summarize findings of two separate surveys, polling 1,088 Texans and 347 Austinites, respectively, to understand their opinions on CAV technologies and strategies. Ordered probit, interval regression, and other models are estimated to understand the impact of demographics, built-environment, and other attributes on Austinites' and Texans' WTP to add CAV technologies to their vehicles, as well as the adoption rates of shared AVs (SAVs) under different pricing scenarios, AV adoption timings' dependence on friends' adoption rates, and home-location decisions after AVs and SAVs become common modes of transport. The Texas study's results indicate that those who support speed regulation strategies, and have higher household income, are estimated to pay more for all CAV technologies, but older and more experienced licensed drivers are expected to place lower value on these technologies. The Austin study's results indicate that higher-income technology-savvy males, living in urban areas and those who have experienced more crashes, have higher WTP for the new technologies. Moreover, Texans and Austinites share a common perception and expect fewer crashes to be the primary benefit of AVs, with equipment failure being their top concern.Civil, Architectural, and Environmental Engineerin